1. Field of the Invention
The present invention relates to an image processing technique for a pixel-mixed image which has undergone pixel mixture processing and, more particularly, to an image processing system for restoring a resolution of a pixel-mixed image deteriorated by pixel mixture processing at high speed with high precision.
2. Description of the Related Art
Owing to recent development of semiconductor technology, there has been a trend toward higher-resolution image pickup devices.
Although efforts to obtain devices with smaller sizes and higher speeds have been made, it takes much time to read out data from a multi-pixel image pickup device. Also, high-speed readout involves the problem of heat generation and is regarded as difficult to be realized.
Pixel mixture techniques recently have been developed and widely known (Japanese Patent Application Laid-Open Publication No. 2004-180284, Japanese Patent Application Laid-Open Publication No. 2004-312140, Japanese Patent Application Laid-Open Publication No. 2005-107252, Japanese Patent Application Laid-Open Publication No. 2005-109968, and Japanese Patent Application Laid-Open Publication No. 2005-117192). A pixel mixture technique is a technique for mixing a plurality of pixels on an image pickup device and reading out the plurality of pixels as one pixel. Simultaneous readout of a plurality of pixels allows high-speed data readout. Readout of a plurality of pixels after mixture increases a speed at which an image pickup signal is read out but causes less aliasing than simple thinning readout. This significantly improves a per-pixel S/N ratio.
As a Bayer pattern demosaicking method, a method has recently been proposed for obtaining a demosaicking result optimum in terms of least squares without using a repetitive operation (H. J Trussel and Robert E. Hartwing, “Mathematics for Demosaicking”, IEEE Trans. Image Processing, Vol. 11, No. 4, April 2002).
A pixel mixture technique, however, four-pixel mixture processing reduces a resolution to one-fourth, and nine-pixel mixture processing reduces a resolution to one-ninth.
As a method for restoring a resolution deteriorated by the pixel mixture processing, an image restoration method is proposed for performing restoration processing by minimizing energy between images before and after restoration (“Restoration from Pixel Mixture Using Iterative Method”, Symposium on Sensing via Image Information, B-8, pp. 75-78, Jun. 12-13, 2003).
However, since an image restoration method by “Restoration from Pixel Mixture Using Iterative Method” uses a repetitive operation for restoration processing, the method requires a large amount of calculation and much processing time. Accordingly, there is a need for obtaining a solution at high speed without performing a repetitive operation as in a conventional method when performing restoration processing on a pixel-mixed image.